Specializer wave B: regexp family — corpus 484 -> 505 (TASK-53) - #39
Merged
Conversation
…pike plan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t-regex differential, measured before implementation (TASK-53) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…anslation layer, corpus 484 -> 505 (TASK-53)
The engine is regex 1.x with the measured parity recipe (pins-waveB/
re2-vs-rust-regex.json): RegexBuilder::octal(true), default Unicode mode
(unicode(false) BREAKS (?i) folding parity — measured), and a bind-time
pattern rewrite in retrans.rs closing the whole Perl-class gap (\d ->
(?-u:\d) etc, in-class variants included) plus a reject list for the
irreconcilables (\B, (?<name>), duplicate group names, bounds > 1000,
stacked quantifiers — the last silently WRONG in rust, not just
error-shaped different). Replacement templates translate \N -> ${N},
$ -> $$, with RE2's invalid-rewrite quirks resolved at bind (out-of-
range backref = identity; global bad escape = consume-with-prefix).
Program grows a prepare-time regex table (print/parse round-trips it);
ReMatch/ReExtract/ReReplace run on both backends. Frontend serves
regexp_matches (SEARCH) / regexp_full_match / regexp_extract (''-on-miss,
flat 0..9 group check, NULL-group -> '') / regexp_replace (backslash
backrefs, 'g', NULL options -> NULL — the pinned asymmetry); ~ / !~ are
FULL match (NOT the Postgres search — measured, the DuckDB binder error
names regexp_full_match); SIMILAR TO passes the RAW pattern (no %%
translation) full-anchored; star * SIMILAR TO filters names by unanchored
search while NOT SIMILAR TO negates a full match (independent predicates
— pinned asymmetry); bare COLUMNS('re')/COLUMNS(*) expand in declared
order with alias stamping. List-valued forms stay clean-unsupported.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
docs/known-limitations.md is the user-facing contract: every deliberate limitation grouped by kind (the specialization bargain, row-serving scope, type-system boundaries, measured descopes, contract choices), each with its justification and the exact build-time message. tests/ test_known_limitations.py asserts all of them — lifting a limitation breaks a test and forces the doc to change in the same commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ahrzb
approved these changes
Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The regexp family lands:
regexp_matches/regexp_full_match/regexp_extract/regexp_replace, the~/!~operators, SIMILAR TO on values, and the wave-5-deferred star forms (* SIMILAR TO,COLUMNS('re')). Corpus goes 484 → 505 of 678 with zero wrong answers and zero replay FAILs.The engine decision
DuckDB uses RE2; we use the Rust
regexcrate (new dependency) — but only behind a measured translation layer. The pins fleet ran a 98-entry differential battery (DuckDB and rust-regex side by side, identical inputs) and found:\d\w\s\bare ASCII, rust's are Unicode. Fixed by a bind-time rewrite (\d→(?-u:\d)etc., in-class variants included), each verified byte-for-byte.unicode(false)— was measured to break(?i)folding parity (KELVIN sign, sharp-s). Never used.\B(unservable — DuckDB itself dies at runtime on non-ASCII),(?<name>)groups, duplicate group names, repetition bounds > 1000, stacked quantifiers (a*+— rust silently reinterprets, a wrong-answer risk, not an error-shape difference),\u,\Q\E.\N→${N},$→$$) with RE2's bizarre invalid-rewrite quirks resolved at bind time: out-of-range backref = silent full no-op; global bad escape = consume-each-match-emit-prefix.With that recipe, all 98 battery entries are byte-identical or identically-rejected.
Measured semantics worth calling out
~is FULL match in DuckDB, not the Postgres search — the binder error literally namesregexp_full_match. Implementing from Postgres intuition would silently flip results.'hello' SIMILAR TO 'h%o'is FALSE (%is a literal),'h.llo'is TRUE (regex dot live). It is sugar for regexp_full_match on the raw pattern.regexp_extractreturns''on no-match — never NULL — and its group index is a flat 0..9 check unrelated to the pattern's group count.regexp_replace's NULL-options argument makes the result NULL, while the other functions error on NULL options — a pinned asymmetry.* SIMILAR TO(unanchored name search) and* NOT SIMILAR TO(NOT full-match) are not complements — implemented as independent predicates per the pins.Infrastructure
Programgrows a prepare-time regex table (round-trips through the IR text format; the verifier checks indices and rewrite presence).ReMatch/ReExtract/ReReplacerun on both backends — interp closures own their compiledRegex, cranelift reads a compiled table through the row context. Constant patterns compile at prepare, matching DuckDB's pinned bind-time error eagerness; column patterns stay clean-unsupported.Verification
docs/superpowers/specs/2026-07-27-waveB-regexp-pins.md)🤖 Generated with Claude Code